Driver prediction to improve interaction with in-vehicle HMI

Recently there has been a trend toward increasing the capability of the in-vehicle interface in terms of access to information and complex controls. This has been accompanied by an increase in the complexity of the car Human Machine Interface [HMI]. At the same time, studies have shown that driver distraction can contribute to accidents. This paper provides some possible ways to reduce driver cognitive load by augmenting the interface. We use prediction of the drivers next action or intention in order to provide UI affordances for more quickly selecting actions. Two examples of this are presented: prediction of driver interaction with the car HMI based on the driving history, and prediction of driver intention from the driver speech. In the first example, we used signal processing techniques to extract meaningful features from vehicle CAN and history data, and then we used machine learning techniques to predict the drivers next action. In the second example, we used ASR and natural language processing to extract text features from driver speech, and predict user intention using a neural network and word embedding. The proposed prediction methods for user actions and intentions can be used to improve in-vehicle task performance. 2015 DSP for In-Vehicle Workshop This work may not be copied or reproduced in whole or in part for any commercial purpose. Permission to copy in whole or in part without payment of fee is granted for nonprofit educational and research purposes provided that all such whole or partial copies include the following: a notice that such copying is by permission of Mitsubishi Electric Research Laboratories, Inc.; an acknowledgment of the authors and individual contributions to the work; and all applicable portions of the copyright notice. Copying, reproduction, or republishing for any other purpose shall require a license with payment of fee to Mitsubishi Electric Research Laboratories, Inc. All rights reserved. Copyright c © Mitsubishi Electric Research Laboratories, Inc., 2015 201 Broadway, Cambridge, Massachusetts 02139 The 7th Biennial Workshop on Digital Signal Processing for In-Vehicle Systems, Oct. 14-Oct. 16, 2015, San Francisco, CA Driver prediction to improve interaction with in-vehicle HMI Bret Harsham, Shinji Watanabe, Alan Esenther, John Hershey, Jonathan Le Roux, Yi Luan, Daniel Nikovski, Vamsi Potluru : MERL, USA E-mail: {harsham,watanabe,hershey,leroux,nikovski}@merl.com : The MathWorks, USA E-mail: [email protected] : University of Washington, USA E-mail: [email protected] : Comcast Labs, USA E-mail: vamsi [email protected] : Alan Esenther, Yi Luan, and Vamsi Potluru contributed to this work while at MERL. Abstract Recently there has been a trend toward increasing the capability of the in-vehicle interface in terms of access to information and complex controls. This has been accompanied by an increase in the complexity of the car Human Machine Interface [HMI]. At the same time, studies have shown that driver distraction can contribute to accidents. This paper provides some possible ways to reduce driver cognitive load by augmenting the interface. We use prediction of the driver’s next action or intention in order to provide UI affordances for more quickly selecting actions. Two examples of this are presented: prediction of driver interaction with the car HMI based on the driving history, and prediction of driver intention from the driver speech. In the first example, we used signal processing techniques to extract meaningful features from vehicle CAN and history data, and then we used machine learning techniques to predict the driver’s next action. In the second example, we used ASR and natural language processing to extract text features from driver speech, and predict user intention using a neural network and word embedding. The proposed prediction methods for user actions and intentions can be used to improve in-vehicle task performance.Recently there has been a trend toward increasing the capability of the in-vehicle interface in terms of access to information and complex controls. This has been accompanied by an increase in the complexity of the car Human Machine Interface [HMI]. At the same time, studies have shown that driver distraction can contribute to accidents. This paper provides some possible ways to reduce driver cognitive load by augmenting the interface. We use prediction of the driver’s next action or intention in order to provide UI affordances for more quickly selecting actions. Two examples of this are presented: prediction of driver interaction with the car HMI based on the driving history, and prediction of driver intention from the driver speech. In the first example, we used signal processing techniques to extract meaningful features from vehicle CAN and history data, and then we used machine learning techniques to predict the driver’s next action. In the second example, we used ASR and natural language processing to extract text features from driver speech, and predict user intention using a neural network and word embedding. The proposed prediction methods for user actions and intentions can be used to improve in-vehicle task performance.